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Abstract 

In this note we study the convergence of the survey decimation algorithm. An analytic 
formula for the reduction of the complexity during the decimation is derived. The 
limit of the converge of the algorithm are estimated in the random case: interesting 
phenomena appear near the boundary of convergence. 

1 Introduction 

Recently a very powerful algorithm has been proposed Ul U\ for finding the solution of the 
random K-satisfiability problem |S1 111 Ej- This new algorithm (see also [HI 13 El El ED]) is 

based on the survey-propagation equations that generalize the older approach based on the 
"Min-Sum" ^ algorithm [m US CSl d • 

The aim of this note is to progress in the understanding of the deep reasons of the very 
good performance of this survey decimation algorithm. In the second section of this note we 
present a fast heuristic derivation of the survey equations. In the third section we analyze 
the decimation algorithm and we give an analytic formula for the decrease in the complexity 
during the decimation. Finally, in the fourth section we present some numerical studies of 
the decimation algorithm in the random case: they suggest an upper bound on the region 
where the algorithm may converge; we notice the appearance of new phenomena near the 
boundary. 

^The "Min-Sum" is the the zero temperature limit of the "Sum-Product" algorithm and sometimes is 
also called belief propagation. In the statistical mechanics language the belief propagation equations are 
the extension of the TAP equations for spin glasses and the survey-propagation equations are the TAP 
equations generalized to the broken replica case. 
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2 A fast heuristic derivation of the survey equations 



2.1 The random K-sat problem 

In the random K-sat problem there are variable a{i) that may be true of false (the index 
i will sometime called a node). An instance of the problem is given by a set of M = aN s. 
For K = 3 each clause is characterized by a set of three nodes (^l,^2^ "is), that belong to the 
interval 1 — and by three Boolean variables (/3i,/32, Ps)- In the random case the i and b 
variables are random with flat probability distribution. Each clause c is true if the expression 

E, = {a{tl) XOR OR (a(i^) XOR f3l) OR XOR (51) (1) 

is true. The problem is satisfiable iff we can find a set of the variables a such that all the 
clauses are true. The entropy ^1] of a satisfiable problem is the logarithm of the number of 
the different sets of the a variables that make all the clauses true. 

To a given problem we can associate a graph (the factor graph 15j) where the nodes are 
connected to the clauses (3a in average) and each clause is connected to three nodes. The 
properties of this graph play a very important role. Some of the considerations we are going 
to use in the following we be valid in the random case, where when A^ — > cxd at fixed a the 
factor graph is locally a tree. 

2.2 Beliefs, warning and surveys 

Generally speaking if the problem is satisfiable, very often there are many configurations of 
the boolean variables that satisfy it. One would like to have some description of the set 
of configurations that satisfy the all the clauses (in the rest of this paper we will call the 
configurations that satisfies all the clauses legal configurations). 

In the simplest approach one introduces the strong belief or warning variable h{i). They 
may takes tree values: true, false or unknown (in the context of colorability [Sj we can 
introduce a new color: white pill lUj). 

In an heuristic approach one assumes that for certain values of the parameters the set 
of legal configurations may be decomposed into sets such that each element of the set is 
near to the other elements of the set and is far the elements of the other sets. 

For each set we define the warning ^ corresponding to a given set according to the following 
rule: 

• If a{i) is true in all the legal configurations of the set C^, h^{i) is true. 

• If a{i) is false in all the legal configurations of the set C^, h^{i) is false. 

• If a{i) is true in some legal configurations of the set and it is false in some legal 
configurations of the same set, h.~f{i) is unknown (or indifferent). 

^The usual beliefs at the node z is a variable P'j{i) that represent the probability that the variable is 
true in a randomly chosen legal configurations of the set C^. Obviously the warning h^(i) is true if p^{i) — 1, 
is false if ^^(z) = and is unknown if < p^(i) < 1. 
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One can also introduce directional warning {b.y{i, c)): they are defined to be the strong beliefs 
at i in absence of the clause c (we consider only the case where i E c). 

Using this definition of warning it can be argued that in the limit N —>■ oo for a random 
problem the directional warnings satisfy (or quasisatisify) the warning propagations equa- 
tions (that we will write later). It is possible to argue that we can associate to any legal 
configuration a solution (or a quasi-solution) of the warning propagations equations, so that 
the legal configurations can be divided into clusters according to the solution of the warning 
propagation equations they correspond to [TUj . 

A very important quantity is the number of solutions of the warning equations that 
correspond to some legal configuration (such a belief will be called a legal warning). This 
number is given by exp(S), where S is called the complexity of the problem. 

We would like to compute the complexity E and get some information on the structure 
of the warnings. It argued that this can be done in the following way ^ |2] . One introduces 
the survey s{i) = (^^(i), si{i), sp{i)) that is a a three component vector: the probability that 
in the set of legal strong beliefs b^{i) is true, indifferent and false is given by sxii), sj{i) 
and spii) respectively. In a similar way we introduce the directional survey s{i, c) that is the 
survey at / with the clause c removed. Obviously a survey satisfies a normalization condition: 

st + si + sf = 1 . (2) 
It can be argued that for large the complexity can be approximatively written as 

PI21II71IIH1. 

S = E ^N{^) - E iK{c) - l)Sc(c) (3) 

i=l,N c=l,M 

where K{c) is the number of boolean variables that enters in the clause c. We have also 
defined 

Sc(c) = ln((l- n if3:-s{ta,c))p] , (4) 

where the product of a boolean variable (3 with a survey s is s itself if (3 is true and it is 
given by the vector sp,si,st if the variable /3 is false (sp denotes the third components of 
the vector s). 

The definition of Sc(i) is slightly more involved. It is given by 



Siv(i) = In 



l[u{i,c] 



(5) 



where u{i, c) is a message from a clause to a node and we have defined the product of two 
vectors in the following way 

VW = {vt Wt + Vj Wt + Vt Wi , Vj Wj , Vp Wp + Vj Wp + Vp Wj}. (6) 

The vector (0, 1, 0) is the identity. The norm \v\ of the vector a is defined by 

\v\ = Vt + vj + Vp . (7) 
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Surveys have norm 1. 

We still have to define the message from a clause to node {u{i,c)). It is a normalized 
vector |'u(i,c)| = 1 fixed by the following condition : 

Ec(c) =\n\u{i,c)s{i,c)\ , (8) 

where |u(z,c)| does not depend on s{i,c). In the case where all the b variables are true and 
an explicit computation gives 

n(z,c) = (/,l-/,0) , /= n s{za,c))F. (9) 

a=l,K{c),ia^i 

The quantity and Sc(c) have the meaning of the variation of the complexity when we 

add the node i and the clause c respectively. 

Heuristically one suppose that the surveys satisfy the survey propagation equations that 
are defined to be the stationary equation of the complexity. 

: (10) 



They can be written in an more explicit form as 



The survey sii) is given by the relation 

s{i) oc u{i^ d) oc s{i, c)u{i, c) Wc E i. . (12) 

rfei, 

It is interesting that the warning equations have exactly the same form of the survey 
equations if we assume that the surveys may be only one of the following three forms: (1, 0, 0), 
(0, 1, 0) and (0, 0, 1). Generally speaking we will only consider in the following solutions of 
the surveys equations that are not of the previous form. 

All this is heuristical. Independently of the derivation of the survey equations its inter- 
esting to study their properties on a random lattice. Numerical experiments and analytic 
computations [TJ suggest that in the limit N ^ oo the survey equations have an unique 
non-trivial solutions (i.e. different from the trivial solution s{ia,c) = I) in the interval 
aL < a < («L and au are are near to 3.91 and 4.36) and this unique solution may be 
obtained by iterations. In this interval the complexity is a decreasing function of a that 
changes sign at a* ~ 4.267. 

The interval ai < a* is interesting because here simple methods have difficulties in find 
a legal configuration. For a given problem in the interesting region we can find solutions to 
the survey equations. This solution carries information on the on the legal configurations so 
that it is natural to try to use them in an algorithm to find legal configurations. 
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Figure 1: The complexity density S as function of the fraction of decimated nodes in the 
region of positive complexity for a problem with N = 3 10^ for three values of a (i.e. 4.2. 
4.25, 4.26 from above to below). 

3 Survey inspired algorithm 

The basic hypothesis beyond the survey decimation algorithm is that the solution of the 
survey equations give reliable information on the problem. In particular we assume that: 

• If the complexity is positive, there exist legal configurations and the problem is satis- 
fiable; if the complexity is negative, there are no legal configurations and the problem 
is not satisfiable. 

• If the survey equations have only the trivial solution {s{ia, c) = /), the problem is easy 
and it can be easily solved. On the other hand if the survey equations have a non trivial 
solution with positive complexity, the problem has solutions but they may be difficult 
to be found. 

Here we are taking for granted these two hypothesis. Our aim is to simplify the problem 
in such a way that it becomes an easy problem. This will be done using the decimation 
algorithm introduced in pi (2J and described below. 

3.1 Decimation 

If a survey (sj) is very near to (1,0,0) (or to (0,0, 1)) in most of the legal solutions of the 
warning equations (and consequently in the legal configurations) the corresponding local 
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variables will be true (or false). 

The main step in the decimation procedure consists is starting form a problem with 
variables and to consider a problem with — 1 variables where s{i) is fixed to be true (or 
false). We denote 

A(i) = - S^-i . (13) 

If A(i) is small, the second problem it is easier to solve: it has nearly the same number of 
solutions of the warning equations an one variable less. (We assume that the complexity can 
be computed by solving the survey equations.) 

The decimation algorithm proceeds as follows. We reduces by one the number of variables 
choosing the node i in the appropriate way, e.g. by choosing the variable with minimal A(i). 
We recompute the solutions of the survey equations and we reduce again the number of 
variables. At the end of the day two things may happen: 

• We arrive to a negative complexity (in this case we are lost), 

• The non trivial solution of the survey equation disappears. If this happens the reduced 
problem is now easy to be solved ^. 

This program may be successfully if we have a good criterion for choosing the point i and 
estimating A{i). Intuitively one would expect that if s{i) = {st,sj,sp) and st is large we 
have that 

A(i) = -ln(l - sf) . (14) 

Indeed if we fix the variables a{i) to be true, we loose all the solutions of the warning equations 
such that b{i) is false and therefore the total number of solutions of the warning equations 
should decrease by a factor {1 — sp)- 

We now want to prove that this intuitive argument is correct. More precisely let us define 
the certitude of a survey as 

sc = max(si;' + sj, st + sj) = 1 — min(sT, sp) . (15) 

If sc{i, c) = 1 — e, we have that 

A{t) = \n{l-sc{t)) + 0{e') . (16) 

The crucial step consists in observing that the survey equations of the system with — 1 
variables are the same of those of variables if we do not use the equations for s{i, c) and 
we set s{i, c) = (1, 0, 0) = T. Let us call s* the solutions to these new survey equations. In 
general s* — s = 0(e). The stationary equations imply that E^(s^ — S^(s*) = 0(e^),. At 
the end of the computation, we find that neglecting terms of order we have: 

A{t) = J:N{^) -Y.H^u{t,c)) . (17) 

■^It is also possible that the surveys propagation equations do not have anymore a solution that can be 
reached by iterations (e.g. if the denominator in eq. Illl is zero: apparently this happens only in the negative 
complexity region. 
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Figure 2: The initial (filled circles) and the final (empty circles) complexity density for a 
problem with = 3 10^ as function of a. The lines are a linear fit. 

A detailed computations shows that the r.h.s. of the previous equation is just given by 
In(l-sc). 

If we neglect the terms of order e^, the previous argument suggests the that the best 
variable to be eliminated are those that have the higher certitude, or the smallest value 
min(sr, sp). This last criterion is similar, although different to the one used in where the 
sites with maximum polarization, i.e. \st — sp\. The two quantities are obviously correlated: 
if the polarization is near to one also the certitude is near to one. 

4 The limits of the algorithm. 

Here in order to illustrate how the algorithm works we report for completeness the results of 
a few numerical experiments we have done on large samples (from N = 10^ to = 3 10^ near 
a* where (for fastening the algorithm) a fraction / = 10~^ of the total variables has been 
decimated simultaneously. In fig. (0) for one sample with A^ = 3 10^ we plot the complexity 
as function of the number of iterations for three different values of a where we have blocked 
the surveys with maximal polarization. We see that for the low value of a the method does 
work, the complexity jumps to zero coming from a positive value, while for the high value of 
a the complexity becomes negative. A very similar is obtained is done in the case where we 
select the surveys using the maximum value of the certitude. 

In fig 121 we plot the complexity density T,m/M (M is the number of undecimated nodes) 
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Figure 3: The quantities 1 — spii) (continues curve) and the quantity A(i), averaged on 
a window of ten decimations (scattered points) of as function of the fraction of decimated 
nodes for one problem with N = 3 10^ and a = 4.2. The variable i is the decimated node. 

at the starting point and at the final point of the decimation procedure. We see that the 
initial complexity density extrapolates to zero at a ^ 4.267 (in perfect agreement with the 
analytic estimates mi2]-) while the final complexity becomes negative at a a ~ 4.252. Similar 
results are obtained for smaller values of A^. 

The conclusion is that in the present form the survey decimation algorithm may work 
in the infinite limit for a < ~ 4.253 < Oc- The numerical experiments seem also to 
indicate that near a a the complexity becomes negative near / = 1. The reasons for this 
remarkable phenomenon will not be discussed here. 

Very similar results are obtained if we use the certitude sc{i) to select the spins: there 
are very minor differences which need a very careful analysis to be evidenziated at least if we 
are not to near to a^, that may slightly depends on the method used. En passant we have 
also verified that the quantity sc{i) is strongly correlated with A[i) and the high order terms 
in e are nor very important. In fig. Q we see for = 3 10^ and a = 4.2 these two quantities 
as function of /. It is remarkable that in the average these two quantities coincide, i.e. if we 
smooth A{i) on a sufficiently large window it becomes very near to 1 — spii) 

The behaviour of the polarization (i.e. |sr — Si?|) of the chosen variable as function of the 
fraction / of removed variables is very similar to that of the certitude (the two quantities are 
strongly correlated) and it is shown in fig. (0)). 

The behaviour of both quantities is remarkable. The behavior for small / (e.g. / < .02) 
can be easily understood and it can be obtained from the distribution of the surveys of 
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Figure 4: The polarization |st — sf| of as function of the fraction of decimated nodes for 
one problem with = 3 10^ and a = 4.2, 4.25, 4.26 from below to above. 

the undecimated problem. The increase of the polarization of the chosen spin after the 
minimum around / <~ 05 is an effect of computing the solution of the surveys equation in 
the decimated problem. It is a very interesting phenomenon and it is at the root of the good 
performances of the survey decimation algorithm. 

The numerical experiments seem also to indicate that near the value of / where the 
complexity becomes negative goes to one, al least with the algorithm where the decimated 
clause has the maximal certitude. In fig. © we see the complexity as function of / for a 
sample with = 3 10^ and a = 4.2525. Here the complexity jumps to zero at / = .993. 
However one should do a more careful and accurate finite size analysis data to see how this 
effect depends on the algorithm and on the sample. 

Let us just sketch a simple intuitive argument for explaining this behaviour of the system. 
Let us assume that: 

• The complexity can jump to zero when the non-trivial solution of the survey disappear 
only if the value of the complexity is near to zero or negative. 

• The probability for the decimation process to be stopped by the presence of a zero in 
the denominator of eq. ^2 is is small for small E and it has a natural prefactor that 
diverges when / goes to one. 

• At fixed / the complexity is a decreasing function of a: a) /da < 0. 
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Figure 5: The complexity as function of the fraction / fraction of decimated nodes for one 
problem with = 3 10^ and a = 4.2525 The complexity jumps to zero at / = .993 where 
the number of undecimated clauses is about 2000. 

If the maximum value of / would be less than one at a a, we would find a contradiction in 
the behaviour at a slightly greater that a a: the survey decimation process would end with 
a jump from a negative complexity and this is prohibited. The contradiction would not be 
present if the maximum value of / is 1, because the stopping probability diverges here. 

In order to explain the performances of the algorithm it would important to find a more 
direct argument that the maximum value of / is 1 at etc- 



5 Conclusion 

The main result of this paper is the identification of the quantity (i.e. the certitude 1 — 
min(sr,Si?)) that controls the complexity reduction during the decimation and the iden- 
tification of the threshold value of oa where the decimation algorithms must stop to work. 
Numerical simulations indicate that interesting phenomena happens near a a, however a more 
careful investigation is needed in order to properly quantify them. An analytic understanding 
of these phenomena is lacking at the present moment: it would be very important to obtain 
it because it would a key step in understanding the reasons for the good performances of the 
survey decimation algorithm. 
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